Threats to Validity When Using Open-Ended Items in International Achievement Studies: Coding Responses to the PISA 2012 Problem-Solving Test in Finland

Autor/in	Arffman, Inga
Titel	Threats to Validity When Using Open-Ended Items in International Achievement Studies: Coding Responses to the PISA 2012 Problem-Solving Test in Finland
Quelle	In: Scandinavian Journal of Educational Research, 60 (2016) 6, S.609-625 (17 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0031-3831
DOI	10.1080/00313831.2015.1066429
Schlagwörter	Achievement Tests; Foreign Countries; International Assessment; Secondary School Students; Coding; Academic Achievement; Content Analysis; Problem Solving; Audio Equipment; Item Analysis; Statistical Analysis; Semi Structured Interviews; Test Validity; Test Reliability; Finland; Program for International Student Assessment + Suchen Sie Ihr Suchwort? Achievement test; Achievement; Testing; Test; Tests; Leistungsbeurteilung; Leistungsüberprüfung; Leistung; Testdurchführung; Testen; Ausland; Sekundarschüler; Codierung; Programmierung; Schulleistung; Inhaltsanalyse; Problemlösen; Audio-CD; Itemanalyse; Statistische Analyse; Testvalidität; Testreliabilität; Finnland
Abstract	Open-ended (OE) items are widely used to gather data on student performance in international achievement studies. However, several factors may threaten validity when using such items. This study examined Finnish coders' opinions about threats to validity when coding responses to OE items in the PISA 2012 problem-solving test. A total of 6 discussions during 6 coder practice sessions (on 6 OE items) and an interview between 5 coders were audiorecorded and analyzed by means of content analysis, and 3 main threats to validity were found: (1) unclear and complex questions; (2) arbitrary and illogical coding rubrics; and (3) unclear and ambiguous responses. Suggestions are given as to how to respond to these threats in order to improve the validity of international achievement studies. (As Provided).
Anmerkungen	Routledge. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site: http://www.tandf.co.uk/journals
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2020/1/01